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WEATHER EFFECTS GENERATOR FOR SIMULATION SYSTEMS 



T Br wwTCAT, *"r*!T.n nv THE INVENTION 

This invention relates to weather simulation, and more 
particularly to a weather effects generator that generates 
a weather database from real world weather data and 
prepares the data for use by a simulator. 
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Weather simulation is often associated with visual 
displays used for aircraft flight training. such systems 
typically provide "out-the-window" cockpit displays of the 
terrain that would be seen during actual flight. m 
addition to terrain, an enhancement to flight training 
systems is the simulation of weather conditions as part of 
the out-the-window display. other applications, such as 
weather reporting, might also use computer-generated 
weather displays. 

In addition to providing visual displays of weather 
conditions, weather simulation can be used in other 
simulation systems. For example, a simulation of a 
structural design might include the effect of wind on the 
structure. or, a simulation of flight instrumentation 
might include a display of the output of weather radar. 

Most existing simulation systems use computer- 
generated weather data. However, U.S. Patent Serial No. 
08/326601, entitled "Heather Simulation System", by Montag, 
et al, and assigned to Southwest Research Institute, 
describes the use of real-world weather data in a 
simulation system. Three-dimensional weather data is 
obtained from real-world sources such as that detected by 
radar or satellite instrumentation. This source data is 
processed so that it can be coordinated with simulation 
data of a simulation system. For example, in the case of 
a flight training system, the aircraft can be located in a 
scene that illustrates real-world weather conditions. 
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fiTTMMAPV OF THE INVENTION 

One aspect of the invention is a weather effects 
generator that provides weather data to a simulator, with 
the weather data being based on real-world weather data. 
5 In first application, the weather data is for use by an 

image generator that provides a visual display of weather 
conditions. In a second application, the weather data is 
for use by a sensor display simulator. In a third 
application, the weather data is for use by a simulator of 
10 an observed object affected by the weather. The weather 

data can also be provided to a multi-function simulator 
that performs any combination of these applications. 

Thus, one aspect of the invention is a method of 
providing data to an image generator that generates a 
15 three-dimensional infrared image. A real-world weather 

database is accessed to obtain a three-dimensional set of 
data elements. Each data element has at least a location 
value, a liquid water content value, and a temperature 
value. The data elements are culled to determine which are 
20 within a f ield-of-view, to obtain a set of f ield-of-view 

data elements. Then, the f ield-of-view data elements are 
sorted to form a list of data elements in depth order. A 
graphics primitive is assigned to each f ield-of-view data 
element. The graphics primitives associated with the 
25 frontmost data elements are used to cover an image plane, 

such that a certain percentage of the image plane is 
covered. This covering step is repeated, using graphics 
primitives in front to back order, until the image plane 
has been covered a predetermined number of times or until 
30 a predetermined number of f ield-of-view data elements have 
been used. The f ield-of-view data elements are then 
assigned to one or more depth bins on the basis of the 
results of the covering step, so as to generate a 
prioritized display list. 
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For simulators having a moving f ield-of-view or a 
changing location, the source data is processed on an "as 
needed" basis, so that only the weather within a current 
f ield-of-view or location is processed. Various 
5 preprocessing and data handling tasks minimize the amount 

of data to be processed so that real time displays can be 
generated without significant impact to the simulator's 
timing or sizing characteristics. 
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RPTBy DES CRIPTION OF THE DRAWINGS 

Figure 1A is a block diagram of a weather effects 
generator in accordance with the invention. 

Figure IB is a block diagram of the various simulators 
5 , with which the weather effects generator of Figure 1 may be 
used. 

Figure 2 is a block diagram of a computer system for 
implemented the weather effects generator of Figure 1 and 
for executing the method of the invention. 
10 Figure 3 illustrates a portion of a geographic area 

whose weather data is stored in the unified weather 
database of Figure 1. 

Figure 4 illustrates a portion of the dataspace of 
Figure 3, whose data elements represent a cloud formation 
15 illuminated by the sun. 

Figure 5 illustrates a f ield-of-view within a 
dataspace • 

Figure 6 illustrates the spatial distribution of 
culled and sorted data elements. 
20 Figure 7 illustrates how the f ield-of-view is divided 

into viewing bins. 

Figure 8 illustrates a viewing plane with its viewing 
bins being covered by splat-type graphics primitives. 

Figure 9 illustrates a viewing plane covered with 
25 billboard-type graphics primitives. 

Figure 10 illustrates how parameters of data elements 
are mapped to textures for graphics primitives. 

Figure 11 illustrates a weather radar cockpit display 
generated by a sensor display simulator with data provided 
30 by the weather effects generator of Figure 1. 

Figure 12 illustrates a wind display generated by a 
sensor display simulator with data provided by the weather 
effects generator of Figure 1. 
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Figure 13 illustrates an air speed gauge generated by 
an object simulator with data provided by the weather 
effects generator of Figure 1. 
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myr&TT.Bn r)Pgn?TPTTnH oy THF TNVENTION 

Figure 1A is a block diagram of the processing and 
data storage components of a weather effects generator 10 
in accordance with the invention. In general, weather 
5 effects generator 10 uses source data from a real-world 

weather data source 11 to generate data that can be used to 
display weather conditions or to simulate weather effects. 
The components of weather effects generator 10 reside in 
the memory of a computer system, and when executed, 
10 implement the method of the invention. 

weather effects generator 10 is especially suitable 
for providing weather data to a host simulator. The 
particular characteristics of the data and the spatial 
distribution of the data depending on the type of 
15 simulation that is to be performed. 

Figure IB illustrates various types of host simulators 
with which weather effects generator 10 may be used. 
Referring to both Figures 1A and IB, these include four 
general types of host simulators 12a - 12d. Each simulator 
20 12a - I2d receives appropriate data from weather effects 

generator 10 and performs the functions required to use the 
data for display cues. 

A first type of host simulator is an image generator 
12a. Such a simulator 12a is described in U.S. Patent 
25 serial No. 08/326601, entitled "Weather Simulation System", 

incorporated by reference herein. This patent application 
describes the structure and operation of a weather 
simulator for illustrating the weather of a scene perceived 
in visible light. A specific example of image generation 
30 is an "out-the-window" display of the weather as viewed 

from the cockpit of a simulated aircraft. A flight 
simulator receives weather data from weather effects 
generator 10 and generates a realistic display of clouds, 
fog, brightness, and other weather conditions. Another 
example of image generation is a simulation of an infrared 
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display. For image generators 12a , weather effects 
generator 10 provides a prioritized display list 
representing real-world weather conditions. 

An example of a sensor display simulator 12b is a 
5 % radar display simulator. For this type of simulator, 
weather effects generator 10 provides a display list of 
water content values having an azimuthal spatial 
distribution corresponding to the radar's range scale. 

An example of a passive object simulator 12c is a 

10 mechanical design simulator that creates a model of a 
structure such as a bridge or a tower that will be exposed 
to weather conditions. For the simulation system, weather 
effects generator 10 provides data representing weather 
conditions that are currently acting on the structure. 

15 Thus, the data is spatially distributed according to the 

current location of the object. 

An example of an intelligent object simulator 12d, is 
an autonomous object simulator that simulates the behavior 
of a moving object that makes decisions about 

20 traff icability conditions such as muddiness or water 

inundation. For this type of simulator, weather effects 
generator 10 provides the same sort of data that is 
provided to the passive object simulator 12c and also 
provides traff icability data. 

25 A feature of the invention is that weather effects 

generator 10 may simultaneously provide weather data to a 
host simulator that has multiple simulation subsystems. In 
this case, the simulation subsystems would be one or more 
of the various types of simulators 12a - 12d. In other 

30 words, multiple simulators 12a - 12d can be coordinated 

with respect to weather conditions and effects. For 
example, a flight training system might have all four types 
of simulators 12a - 12d. The pilot could view an out-the- 
window display of the weather. He might also use an 

35 instrumentation panel with a weather radar display. His 
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aircraft could be subjected to weather conditions such as 
wind, and he could be required to contend with other 
aircraft whose behaviors are modeled as autonomous agents* 
Weather effects generator 10 generates a weather database 
5 from which it obtains appropriate data for each type of 

subsystem. The type of data and the spatial distribution 
of the data varies according to the type of simulation. 

Another feature of the invention is that weather 
effects generator 10 may provide data in the environment of 
10 moving as well as stationary objects. For example, an 
image generator 12a might illustrate the weather from an 
aircraft in flight or from a fixed viewpoint. Likewise, a 
sensor simulator 12b might simulate a sensor on a moving 
object or from a fixed location. The passive or 
15 intelligent objects of simulators 12c and 12d could be in 

motion or stationary. In each case, the weather 
environment changes with time or with location, or with 
both. As the weather changes, weather effects generator 10 
updates the data it provides to the simulator. 
2 0 Often it is desirable for weather effects generator 10 

to provide its weather data to the host simulator 12a - 12d 
so that real time motion can be depicted. An advantage of 
the invention is that it permits such real-time rendering 
of weather effects. The embodiment of this description 
25 assumes a real time display data rate of at least 30 frames 

per second. However, in other embodiments, single images 
or slower than real-time displays could be useful. 

Figure 2 is a block diagram of a computer system 20 
for implementing weather effects generator 10 and for 
30 performing the method of the invention. In general, the 

components of system 20 are conventional computer 
components, and include a processor 21 for executing 
instructions for the interface, preprocessor, and data 
handler functions of Figure 1A. Although Figure 2 is a 
35 system having a single processor for performing the 
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functions of each processing component, the system could be 
physically as well as functionally divided into separate 
processing components. A program memory 22 stores program 
code. A data memory 23 stores data, and includes a mass 
5 memory for storing weather database 15. The system also 

has a graphics display 24 and an input device, such as a 
keyboard 25. A timing unit 26 provides timing signals for 
real-time displays. The present invention relates to the 
use of a computer system, such as that of Figure 2, which 
10 stores and executes programming for weather effects 

generator 10. 

Referring again to Figure 1A, weather database 11 
stores data representing real-world weather conditions. 
This data is "real world" data in the sense that it 
15 represents changing weather conditions as they actually 

occurred in a real-world environment. 

More specifically, database 11 is a three-dimensional 
collection of data elements, each data element representing 
a volume of airspace. Each data element has a set of 
20 weather values, each representing a different parameter. 

One parameter is three dimensional location. The other 
parameters may be one or more of the following weather 
parameters : 

wind speed (x,y,z) (kts) 
25 location (x,y,z) (ft) 

radar reflectivity (DBz) 

pressure (mB) 

temperature (JC) 

water type (fog, hail, rain, snow, etc) 

30 liquid water content (g/» 3 ) 

precipitation rate (mm/hr) 
Wind speed is a vector value, having both magnitude and 
direction. Water type is an indication of whether the 
conditions at a given location are rainy, snowy, icy, etc. 
35 As explained below, although simulators 12a - I2d each 
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require location values, they nay require different weather 
values . 

Radar reflectivity can be used to derive liquid water 
content, and it is possible that database 11 could already 
store liquid water content values. For this reason, radar 
reflectivity values and liquid water content values are 
both referred to herein as "liquid water content" values. 

Database 11 can be collected by known means of 
acquirinq weather-related data. For example, radar 
reflectivity values can be qenerated from weather radar 
output. Satellite-based earth observinq weather systems 
are a source of remotely sensed atmospheric data. It is 
also possible that database 11 could be comprised of a 
combination of real-world and artificially derived 

15 parameters. 

Fiqure 3 illustrates a portion of a dataspace 30, 
which is comprised of weather data for a qeographic area. 
Dataspace 30 is a three-dimensional array of volume data 
elements 31, and is oriented with respect to earth- 
20 referenced coordinates, x,y,z. Althouqh not explicitly 
shown in Fiqure 3, the position and wind coordinates of 
each data element 31 are at a specific x,y,z point in that 
data element 31, such as at its counterpoint. Spacing of 
data elements 31 is typically non-uniform (by dithering) to 
25 provide spatial variation. 

A typical dataspace 30 might represent a real-world 
area of 800 x 800 nautical miles. This area might be 
divided into 2048 data elements 31 in the x and y 
directions, and 15 layers in the z direction. 

The location values of dataspace 30 need not represent 
the real-world location where the weather actually 
occurred. It is only necessary that the order and spacing 
of the data elements 31 be known so that they can be 
located in the coordinate system of simulator 12. However, 
a feature of the invention is that a real-world location of 



30 



35 



WO 96/16379 



PCT/US9S/14919 



12 



data space 30 could be mapped to a real-world location 
depicted by the simulator 12* 

Weather Effects Data for Image Generatorg 
5 In general, for image generators 12a, weather effects 

generator 10 provides a display list of data elements, 
which are prioritized according to f ield-of-view data 
provided by simulator 12a. The data elements are those 
that affect the visual display, which may be a visible 

10 light display or an infrared display. 

User interface 13 receives various types of input from 
a user to control the display. Two types of data that are 
significant to the invention may be user-supplied. One 
type of such data is threshold data, which permits the user 

15 to specify a threshold for one or more of the weather 
parameters of database 11. For example, the user may 
specify that only liquid water content values over a 
certain threshold value are significant. Another type of 
input data is illumination data, such as time-of-day and 

20 day-of-year that are used to determine the visual effect of 

sunlight or moonlight. The processing of both types of 
data is explained below in connection with preprocessor 14. 

Preprocessor 14 is programmed to create a weather 
database 15 from the source data 11. More specifically, 

25 preprocessor 14 performs the tasks of subsampling, 

thresholding, calculating liquid water content, feature 
calculations, coordinate transformation, and dithering. 

As explained below, different simulators may call 
for different feature calculations. 

30 Subsampling is an optional step, which reduces the 

amount of data within database 11 to be processed. For 
example, every n data elements 31 in each direction might 
be transformed into a larger data element that represents 
the same area. For convenience of description herein, 

35 subsampled data elements are also referred to as data 
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elements 31. To preserve as much information as possible, 
various interpolation or averaging techniques may be used 
to obtain new parameters from those of the subsampled data 
elements 31. 

5 * Thresholding is another optional step for reducing the 

amount of data to be processed. A threshold value is 
specified for one or more parameters in database 11. For 
example, radar reflectivity is indicative of cloud 
formation, and any data element 31 whose value is below a 
10 given threshold is assumed to be clear sky and not further 

processed. The default value for thresholding is zero. 

For each data element 31, preprocessor 14 calculates 
its liquid water content, R, from its reflectivity value. 
An example of such as calculation is: 

R = 12^ (1) 



15 



20 



30 



27.4248 



where ref 1 is the reflectivity value for the data element 
31. 

Figure 4 illustrates a portion of dataspace 30, in 
two-dimensions. In the example of Figure 4, thresholding 
is performed with respect to liquid water content. Each 
data element 31 is either in a cloud or not, as determined 
by its liquid water content value. Thus, for example, 
where data element 31(a) has a liquid water content value 
of zero, it is assumed to be clear sky and is not 
25 processed. Data element 31(b) has a liquid water content 

value greater than zero, but thijs value does not exceed the 
threshold. Thus, data element 31(b) is treated as clear 
sky and not processed. Data element 31(c) has a liquid 
water content that is above the threshold. The boundaries 
of that data element 31(c) and of all other data elements 
31 whose liquid water content values exceed the threshold, 
define the boundaries of the cloud to be visually 
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displayed* These are the data elements 31 that are further 
processed after thresholding. 

For coordinate transformation, the location values of 
the data elements 31, as well as their wind vector 
5 coordinates, are transformed into whatever coordinates are 

used by the simulator 12a. For example, to make the data 
consistent with flight training systems, transformation is 
to north-east-down (N-E-D) coordinates. In the N-E-D 
system, sea level is at a down of zero and altitude is in 
10 negative units. North corresponds to a heading of zero 
degrees and east corresponds to a heading of ninety 
degrees. As a result of transformation, dataspace 30 is 
placed in the vicinity of the viewer and is correctly 
oriented. 

15 For visible light displays, feature calculations are 

directed to transparency and color. These calculations 
include calculations involving illumination and intensity. 
Illumination calculations assume all light to be from a 
point source, whose radiation is modelled as parallel rays. 

20 This point source is typically the sun or the moon. In the 

example of this description, the point source is the sun. 
Sun-angle data (time-of-day and day-of-year) determine the 
angle of the sun with respect to a plane representing the 
earth's surface. This sun-angle data may be provided by a 

25 user via interface 13 , or may be automatically provided by 

timing unit 26. Dataspace 30 is assumed to be oriented on 
a plane parallel to the earth's surface. Referring again 
to Figure 4, the sun-angle data have been used to calculate 
an angle, 8, between the light rays and the plane of the 

30 earth. For each data element 31, it can be determined 

whether other data elements 31 are interposed between that 
data element 31 and the sun. 

Each data element 31 is assigned an intensity value by 
first calculating an illumination value, Itm , that is a 

35 product of the illumination value of a data element 31 
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blocking the sunlight and a liquid water content factor- 
An example of this calculation nay be expressed as follows: 



Ilm^ = Ilm n (1 - „ ) (2) 



where n+1 identifies the data element 31 whose value is 
currently being calculated and n identifies a data element 
31 immediately interposed between the sun. The R^ and 
^mm values are user-defined minimum and maximum values. As 
an example, they may be specified as follows: 

R . = 1 (3) 

mm 27.4248 

R -.JS^L (4) 
m " 27.4248 

In this example, is a constant based on a radar 

reflectively value of 65 DB, which is a reasonable maximum 
water content value for rain. The result of the above 
calculation is an attenuated illumination value that is 
based on the extent to which illumination of any data 
element 31 is attenuated by the water content of data 
elements 31 between it and the sun. 

The R value of each data element 31 is used to 

max 

calculate its intensity value, Int, as follows: 

Int = (Ilm * L B ) (1 - L A ) + L A (5) 



where L B and L A are user-defined light brightness and 
light ambience values. 
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In the example of this description, to facilitate 
computer calculations, all values are normalized on a scale 
of 0 - 1. The intensity value, lnt , also will be a value 
ranging from 0-1. 
5 Figure 4 shows intensity values of some data elements 

31. In general, those data elements 31 that represent a 
part of the cloud that faces the sun are bright* Data 
elements 31 that represent a part of the cloud directly 
opposite the sun are dim. "Bright" and "dim" intensity 

10 values are 1 and o, respectively. All other data elements 

31 would have values ranging from 1 to 0. 

As a result of the above intensity calculations, if 
the viewer 1 s line of sight passes through a cloud, the 
effect of attenuation will be seen. The amount of 

15 attenuation at a given data element 31 is related to the 

amount of water in the volume of other data elements 31 
that a light ray must pass through to arrive at the given 
data element 31. 

Each data element 31 is assigned a color value and a 

20 transparency value. For purposes of this example, a 3 2 -bit 

value for each data element 31 is assumed, with 8 bits for 
each RGB (red, green blue) color value and 8 bits for a 
transparency value, A. Each data element 31 is initially 
assigned the same base color value and base transparency 

25 value. These base values are user-defined and are 

normalized on a scale of 0 - l. Thus, for RGB values that 
range from 0 - 255, color values are scaled from 0 - l. As 
an example, where the base color and transparency values 
are both 0.5, each data element's intensity value is 

30 multiplied by 0.5, resulting in color and transparency 

values ranging from 0-1. 

For an infrared image generator 12a, feature 
calculations are directed to transparency and intensity. 
As in the case of visible light displays, transparency is 

35 a function of liquid water content along the line of sight, 
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which in this case is the line of sight of the infrared 
detector. The water attenuates the signal received at the 
infrared detector. Attenuation is also a function of the 
characteristics of the infrared detector. The amount of 
attenuation determines a transparency value at each data 
element 31. Intensity is a function of the radiance 
emitted along the detector's line of sight. The radiance 
emitted at each data element is determined by temperature 
and liquid water content values for a given frequency band. 
Transparency and intensity values can be calculated by 
preprocessor 14 or by data handler 17. Alternatively, 
weather effects generator 10 can deliver the underlying 
data (liquid water content and temperature) to simulator 
12a. For rasterization, the intensity values are mapped to 
monochromatic color values so that the display looks like 
a conventional infrared display. Like the intensity and 
transparency feature calculations, the rasterization can be 
performed by weather effects generator 10 or the underlying 
feature data can be delivered by weather data generator 10 
20 to image generator 12 a. 

For either visible light displays or infrared 
displays, dithering reduces the occurrence of visual 
artifacts in an image. A randomized offset is applied to 
the three-dimensional location of each data element 31. 

The weather database 15 generated by preprocessor 14 
consists of data elements 31, each described by all or some 
of the following parameters. Each type of simulator 12a - 
l2d receives location data. The weather effect parameters 
in database 15 depend on the type of simulator 12a - 12d to 
30 which data is being provided. 

wind speed (north, east, down) (ft/sec) 
location (north, east, down) (ft) 
radar reflectivity (DBz) 
intensity (0-1) 
color <° " 1 ) 



25 
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transparency (0 - l) 

pressure (mB) 
temperature (K) 
liquid water content (g/» 3 ) 
5 water type (fog, hail, rain, etc) 

precipitation rate (mm/hr) 
Database 15 consists of spatially and temporally 
linked files that are retrieved and loaded into active 
memory as needed by simulator 12a during image generation. 
10 The database 15 is "f our-dimensional" in the sense that it 

represents weather conditions in a three-dimensional volume 
over time. it is structured in a gridded Cartesian format. 
Database 15 is "unified 11 in that it permits a common 
numerical description of real-world weather, which may be 
15 shared among different simulators or among simulation 

subsystems of a multi-function simulator. 

Simulator interface 16 receives f ield-of-view data 
from simulator 12a. As explained below, this data is used 
to determine a f ield-of-view within dataspace 30. If the 
20 viewpoint of simulator 12a is in motion, the f ield-of-view 
data is continually updated. 

Data handler 17 retrieves data from database 15 and 
f ield-of-view data from interface 16. In general, data 
handler 17 performs the operations necessary to generate a 
25 prioritized display list of data elements 31 that are in 

the current f ield-of-view. This display list is used by 
simulator 12a to generate a display that appears three- 
dimensional. Data handler 17 is scheduled by timing 
signals to assure periodic image updates and 
30 synchronization with simulator 12a. 

For display list generation, data handler 17 accesses 
database 15 and operates on each data element 31 
individually. It performs several operations with respect 
to each data element 31: culling, sorting, graphics 
35 primitive assignment, and depth bin assignment. 
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Culling determines what data elements 31 are in the 
current field-of-view of simulator 12a. Bach data element 
31 is considered to determine whether it is between a near 
clipping distance and a far clipping distance. Those that 
are within this range are further considered to determine 
whether they are within an up-down and left-right range 
with respect to a two-dimensional image frame. 

Culling operations may be performed with vector 
projection techniques, using a viewpoint vector, VP, as a 
reference. If the viewpoint changes, the culling process 
is updated. If a data element 31 is not in the field-of- 
view, further processing need not be performed. 

Figure 5 illustrates dataspace 30 as a series of 
transparent planes. The field-of-view is illustrated by 
dashed lines. The intersection of dataspace 30 and the 
field-of-view determines which data elements 31 are within 
the field-of-view. 

For sorting, data handler 17 determines how to 
traverse the database 15 so that the data elements 31 are 
accessed in an order that is based on the distance between 
the data element 31 and the viewpoint. Data elements 31 
that are closest to the viewpoint are processed first. 

As an example of one method of sorting, data elements 
31 that are within the field-of-view are sorted by means of 
three nested loops. The primary sort is back-to-front, and 
the secondary sorts are left-to-right and top-to-bottom. 
Of course, the order of the primary sort could be front-to- 
back, with the "beginning" of the list at its end. In 
either case, the primary sort is by depth. The order of 
30 the secondary sorts could be right-to-left or bottom-to- 

top. 

Figure 5 illustrates a back-left-top priority of 
sorting. The first sort orders the data in back to front 
order, a second orders it left to right, and a third orders 
35 it top to bottom. Other sorting methods could be used, 
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with the result being a prioritized list of data elements 
31 , based on their location in the f ield-of-view, from 
which each data element 31 can be accessed according to its 
depth and from which all data elements 31 of the same depth 
5 can be accessed. 

Figure 5A illustrates the spatial distribution of the 
culled and sorted data elements 31. Each data element 31 
is illustrated in accordance with its wind vector. Only 
those data elements 31 having a liquid water content above 
10 a threshold have been illustrated, and are those data 

elements 31 that will be represented as clouds. The data 
elements 31 are drawn in perspective, such that data 
elements 31 that are closer are larger and more widely 
spaced. 

15 A next step of display list generation is the 

assignment of a graphics primitive to each data element 31. 
One type of graphics primitive that may be assigned to the 
data elements 31 is a polygon-based primitive known as a 
"splat". In mathematical terms, a splat is a three- 

20 dimensional point spread function about the data element 

31. A resulting two-dimensional splat shape is assigned to 
each data element 31, based on a transformation of that 
data element's wind vector onto the image plane. 

More specifically, the three-dimensional point-spread 

25 function applied at each data element 31 is implemented as 

a three-dimensional ellipsoid, such that the longest axis 
through the ellipsoid is coincident with the direction of 
the associated wind vector. The splat shape varies from 
round to elliptical depending on the magnitude of the wind 

30 vector. The greater the wind speed, the more elliptical 

the splat. The length of the longest axis is determined by 
the magnitude of the wind vector and a scale factor. The 
lengths of the other two axes are set to some value that is 
a portion of the length, typically one-half. During image 

35 generation, the ellipsoid, oriented along the wind vector, 



WO 96/16379 



PCTAJS95/14919 



21 



10 



15 



is projected onto the image plane as a two-dimensional 
ellipse. As the viewpoint vector changes, the ellipse is 
rotated so that it maintains a desired relationship between 
its wind vector and the viewpoint vector. 

Figures 6A - 6C illustrate three different splats. 
The data element 31 whose splat is illustrated in Figure 6A 
has a wind vector value of zero, thus its splat is round. 
The data element 31 whose splat is illustrated in Figure 6B 
has a higher wind velocity value than the data element 31 
whose splat is illustrated in Figure 6C. The data elements 
31 whose splats are illustrated in Figures 6B and 6C have 
wind vectors in the same direction. It should be 
understood that Figures 6B and 6C are two-dimensional 
representations of a three dimensional wind vector, thus 
the wind vector's apparent magnitude is a projection of the 
true vector into the view area. 

After each splat is constructed, it is colored and 
illuminated according to the parameters of its associated 
data element 31. The illumination is determined by the 
transparency value, alpha, calculated during preprocessing. 
A maximum alpha value, typically the same as the 
transparency value, is set for the center of the splat 
shape. The edges of the splat have zero alpha values. 
Each splat has alpha values that vary across it as a 
Gaussian function, with the largest value in the center and 
tapering to a zero value at the edges. 

The splat shape along with its varying alpha function 
is the "footprint" of the particular splat. For 
perspective imaging, the footprint for the splat of each 
30 data element 31 is scaled by its distance from the 
viewpoint. Thus, splats that are farther away from the 
field-of-view origin will appear smaller. 

Another type of graphics primitive that may be 
assigned to the data elements is a "billboard". Like 
35 splats, billboards are polygon-based primitives, but have 
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a rectangular shape. Like a splat, its shape and 
orientation are determined by the wind vector. 

Figure 9 illustrates how billboards are used to cover 
the image plane. Each billboard is aligned with the wind 
5 vector of the associated data element 31. In a manner 

similar to that used for splats, billboards are assigned to 
data elements 31 in front to back order, beginning at one 
end of the prioritized display list. The billboards are 
used to cover the image plane in one or more depth bins. 
10 Once billboards are assigned, image generation occurs in 

back to front order. 

Billboards are used in conjunction with a texture 
) library. A specific texture is assigned to each billboard, 
as determined by a look-up table or other reference. 
15 Textures are determined by parameters of the data element 

31. For example, for an infrared display, a two- 
dimensional look-up table might have values derived from 
liquid water content and wind speed. in simpler 
embodiments, infrared textures might be based only on 
20 liquid water content. 

Figure 10 illustrates a two-dimensional texture look- 
up table having 16 textures, which are selected in 
accordance with wind and liquid water content parameters. 
For each data element 31, a wind texture value, 7^ , is 
25 determined as follows: 

^ = <W " W^) (K A ) + K B (6) 



, where w is the magnitude of the wind vector of that data 
element 31 and is a user-defined wind threshold. K A 

and K B are user-defined constants, which limit the texture 
30 values to a predetermined range, here 1-4. a 

reflectivity texture value, T m , is determined as follows: 
T** - & - ^ iC A ) + C B (7) 



WO 96/16379 



PCT/US95/M919 



23 



where R is the reflectivity value of the data element 31 
and is a user-defined threshold. C A and C B are user- 

defined constants, which limit the texture values to a 

5 range of 1 - 4. 

For a data element 31 whose texture values were 

T M*nd = 2 5 and T "fl = 3 5 ' texture T3 would be assigned to 
the billboard for that data element 31. The range of 
values for each texture type corresponds to number of 
10 available textures, here 4X4. high wind and reflectivity 
values are associated with rain-type textures, such as T4. 
Low values are associated with clearer textures, such as 
T13. 

Other textures could be derived from other parameters, 
15 such as pressure and temperature. As an example, 
temperature and pressure could be considered ambient, such 
that different look-up tables could be used for different 
pressure and temperature ranges. 

Texture levels of detail are maintained in the library 
20 to support viewing resolution as a function of the location 
of weather data elements 31. Data elements 31 closer to 
the viewpoint are mapped with higher detail textures than 
those that are far away. Thus, the look-up table could be 
three-dimensional, with the third dimension being a range 
25 value. 

Other than their rectangular shape and texture, 
billboards are processed in the same manner as described 
above for splats. Textures have alpha values that are 
blended as described above. Like splats, billboards are 
30 rotated so that they are always normal to the viewpoint 

vector, VP. 

After graphics primitives are assigned, each data 
element 31 is considered for filling "depth bins", which 
facilitates the rendering of three-dimensional aspects of 
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the image. For this task, the f ield-of-view is first 
divided into "viewing bins". 

Figure 7 illustrates how the f ield-of-view is divided 
into "viewing bins", projected to the viewing plane 70. In 
5 this example, there are four viewing bins, but any number 

could be used. In general, the more viewing bins, the more 
realistically the image can be rendered. 

Each viewing bin is "covered" by locating splats, 
sorted by distance, on the viewing plane and determining 

10 whether the area of the viewing bin is covered. Splats 
whose data elements 31 are closest to the f ield-of-view 
origin have the highest priority. A front-to-back 
traversal through the database 15 is performed to cover the 
viewing bins. In other words, the splats of the 

15 "frontmost" data elements 31 are used first, with the 

covering process being repeated for splats of data elements 
31 of increasing depth. Each splat is assumed to be scaled 
for a perspective view as well as to the dimensions of the 
display, so that a relationship between the dimensions of 

20 each splat and the dimensions of the image plane can be 

determined • 

Figure 8 illustrates the viewing plane 70, with its 
viewing bins being covered by a first layer of splats. The 
splats of the frontmost data elements 31 are being used to 

25 cover the viewing bins first. Furthermore, in accordance 

with a front-top-lef t, prioritized database traversal, the 
splats of top data elements 31 are used before bottom ones, 
etc. The viewing bin in the top left corner is considered 
"covered" because the splat dimensions cover a certain 

30 percentage of its area. 

Each viewing bin is covered in the same manner. If a 
first pass of all data elements 31 of a given depth does 
not cover the image plane 70, another pass is made with 
data elements 31 farther away. Each set of splats that 

35 covers the image plane once is a set of splats for one 
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"depth bin" . This covering process is repeated until the 
viewing bins have been covered a predetermined number of 
times or until all data elements in database 15 have been 
used, whichever occurs first. Splats are added to the 
5 display list if they fall into a depth bin that is not 

full. A depth bin is considered full when it has been 
covered a predetermined number of times. 

The output of data handler 17 is a prioritized display 
list of data elements 31, each associated with a graphics 
10 primitive and a set of data values. This data can be 

supplied to an image generator component of simulator 12a. 
For generating a display, the data in the deepest depth bin 
are used first, then the remaining data, forward through 
the depth bins. A process generally known as 

15 "rasterization- is performed to draw graphics primitives 

with pixel values. Rasterization techniques, as well as 
the use of graphics primitives other than splats, are 
described in U.S. Patent Serial No. 08/326601, incorporated 
by reference above. 
20 . , 

flo ^hpr Effects na t i far Sanaar nisniav simnators 

In general, for sensor simulators 12b, weather effects 
generator 10 accesses the real-world source data, 
preprocesses it, and spatially distributes it. It thereby 

25 provides data elements 31 that represent either a sensed 

weather condition, such as water content for a real-beam 
radar simulator or wind for a simulator that simulates 
sensed Doppler effects, or a weather condition that affects 
the sensor. The preprocessing includes tasks similar to 

30 those described above in connection with image generator. 

Preprocessing may include feature calculations specific to 
the type of sensor, such that the source weather data is 
used to derive data that would be indicated by the 
simulated sensor. 
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For sensor display simulators, data handler 17 
spatially distributes the data elements 31 in accordance 
with the type of sensor. The simulator 12b provides 
location data, which includes information about the 
sensor's range (f ield-of-view) . For example, for two- 
dimensional sensor displays such as a radar display, the 
data elements are ordered in a two-dimensional plane 
consistent with the sensor's range. 

Figure 11 illustrates a radar cockpit display 
generated by a radar display simulator 12b. The data 
elements provided by weather effects generator 10 represent 
water content at a constant elevation, sorted in a two- 
dimensional plane. The spatial distribution is in terms of 
azimuthal range bins. For example, in the case of an 
aircraft radar sensor the azimuthal distribution of range 
bins would represent a wedge-shaped f ield-of-view in front 
of the aircraft. A continuous-scan ground-based radar 
sensor might have a circular f ield-of-view. Simulator 12b 
uses the liquid water content values to determine the pixel 
values of the radar display, where different ranges of 
water content values are represented by different colors. 

Figure 12 illustrates a wind cockpit display generated 
by a wind sensor simulator 12b. The data elements provided 
by weather effects generator 10 represent wind direction 
and velocity at a specified altitude. These values are 
used to calculate a wind factor that indicates whether 
conditions are hazardous. Whereas the display of Figure 12 
is two-dimensional, weather tiata generator 10 could 
alternatively be used to provide a prioritized display list 
for a three-dimensional sensor display. For example, data 
elements 31 could be provided for a three-dimensional wind 
display, using the spatial distribution techniques 
described above in connection with Figures 5-7. 

Another type of sensor simulator 12b to which weather 
effects generator 10 might provide data is a target sensor, 
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such as a millimeter-wave radar, infrared sensor, or 
electro-optical sensor. The ability of these sensors is 
greatly affected by intervening weather. For such 
simulators, weather effects generator 10 provides data 
5 • elements 31 representing weather conditions that cause 
signal attenuation and clutter. The simulator 12b 
incorporates the data element values into its processing to 
calculate radar output as affected by weather related 
effects such as attenuation and backscatter. 

10 m each of the above examples of sensor display 

simulators 12b f the spatial distribution of the data 
elements provided by weather effects generator is 
determined by the range of the sensor being simulated and 
also by whether the display is three-dimensional or two. 

15 In general, simulator 12b will process the data by dividing 

it into some arrangement of "spatial elements". These 
spatial elements might be range bins, such as for radar 
displays, or doppler bins, such as for wind displays. 
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P flff ff iYA and Tn tf 11 iff" 1 *- object Simulation 
For object simulators 12c and 12d, weather effects 
generator 10 access the real-world source data, 
preprocesses data elements 31, and spatially distributes 

25 them. It delivers these data elements 31 to the simulator 

12c or 12d, which uses them to model the effect of weather 
on the object. For example, weather effects generator 10 
might provide wind, pressure, and temperature data so that 
the simulation includes the effects of these conditions on 

30 the object. For a moving object, this process occurs 

continuously as the object moves through the environment 
represented by the database 15. 

For a passive object simulator 12c, weather effects 
generator 10 preprocesses data to determine which data 

35 elements 31 surround the object, using location data 
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provided by simulator 12c. Other preprocessing tasks, such 
as those described above in connection with image 
generation may be performed* 

For object simulation, one type of spatial 
5 * distribution is interpolation, where weather effects 
generator 10 interpolates weather values of these data 
elements 31 at the specfied location. For example, 
interface 16 might interpolate the eight data elements 31 
that comprise a square volume surrounding the object* The 
10 result is a net value of the weather conditions acting on 

the object. 

Figure 11 illustrates an air speed gauge that is part 
of a simulator 12c that simulates an aircraft in flight. 
Simulator 12c uses data provided by weather effects 

15 generator 10 to calculate the ambient and dynamic pressure 

at the aircraft. 

As another example, a simulator 12c might use data 
elements 31 from weather effects generator 10 to modify its 
equations of motion, such as are used to determine speed, 

20 attitude, and orientation of the simulated object. This 

might be interpolated data elements 31 or data elements 31 
that are spatially distributed in some other manner. 

For spatial distribution other than interpolation, the 
simulated object may have either a range or a field-of- 

25 view. These the two terms are equivilent for purposes of 

spatial distribution. In either case, weather effects 
generator 10 receives location data from simulator 12c or 
12d, which includes information about the object's range or 
f ield-of-view. Weather effects generator 10 uses this data 

30 to provide the simulator with data elements 31 that are 

spatially distributed in a manner appropriate for the 
sensor. For example, for a simulated object located in a 
three-dimensional space, the spatial distribution could be 
three-dimensional in an area surrounding the object. 
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For simulator 12d, the simulated object is an 
autonomous agent, which means that it makes intelligent 
decisions. Weather data generator 10 perforins the same 
basic tasks of accessing the real-world data, 
preprocessing, and spatial distribution as for passive 
object simulation. It may provide the same type of data 
elements 31 as for passive object simulation, but in 
addition, it provides data elements 31 having values 
representing weather-related traf f icability. This 
trafficability data may involve feature calculations by 
preprocessor 14 or data handler 17. For example, 
accumulation of liquid water content over time can be used 
to determine how much rain has fallen. Or in the case of 
a boat, wind might determine the water surface conditions. 
15 Such feature calculations are optional in that they may be 

performed by weather effects generator 10 or weather 
effects generator 10 may provide the underlying data to 
simulator 12d. For autonomous objects that "see", visible 
range is determined with the transparency values that are 
20 in the object's f ield-of-view, and weather effects 

generator 10 provides a prioritized display list similar to 
that provided for image generator 12a. 

Where simulator 12c or 12d is used with an image 
generator 12a, the simulated object's behavior is 
25 consistent with what is visually displayed. In other 

words, the data to each type of simulator is from the same 
real-world weather environment. For example, the simulator 
might include a visual display of the simulated object 
undergoing the effects of weather as seen by an observer, 
for example, a bridge reacting to wind. As stated above, 
this is an advantage of the unified database 15, which may 
provide data for different types of simulations. 
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Although the invention has been described with 
reference to specific embodiments, this description is not 
meant to be construed in a limiting sense. Various 
modifications of the disclosed embodiments, as well as 
alternative embodiments, will be apparent to persons 
skilled in the art. It is, therefore, contemplated that 
the appended claims will cover all modifications that fall 
within the true scope of the invention. 
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WHPlT Tg CTATMTin IS; 

1. A method of providing data representing weather 
effects to an image generator that generates a three- 
• dimensional infrared image, comprising the steps of: 

accessing a real-world weather database to obtain a 
three-dimensional set of data elements, each data element 
having at least a location value, a liquid water content 
value, and a temperature value; 

culling said data elements to determine which are 
within a f ield-of-view, to obtain a set of f ield-of-view 

data elements; 

sorting said f ield-of-view data elements to form a 
list of data elements in depth order; 
15 assigning a graphics primitive to each of said f ield- 

of-view data elements; 

covering an image plane with the graphic primitives 
associated with the frontmost of said f ield-of-view data 
elements, such that a certain percentage of said image 
20 plane is covered; 

repeating said covering step, using said f ield-of-view 
data elements in front to back order, until the image plane 
has been covered a predetermined number of times or until 
a predetermined number of said f ield-of-view data elements 

25 have been used; and 

assigning said f ield-of-view data elements to one or 
more depth bins on the basis of the results of said 
covering step, so as to generate a prioritized display 
list. 



2. The method of Claim 1, further comprising the 
step of calculating a transparency value for each of said 
data elements, based on said liquid water content value. 
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3 . The method of Claim 1 , further comprising the 
step of calculating an intensity value for each of said 
data elements, based on said temperature value. 

5 4. The method of Claim 1, wherein said graphics 

primitive is a polygon-based primitive. 



10 



5* The method of Claim 1, wherein said graphics 
primitive is a three-dimensional function, transformed to 
a two-dimensional image plane. 
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6 A method of providing data representing weather 
effects to a sensor display simulator that generates a two- 
dimensional image of wind effects, comprising the steps of: 

accessing a real-world weather database to obtain a 
multi-dimensional set of data elements, each data element 
having a location value and at least one weather effect 
value; 

receiving location data from said simulator; 

culling said data elements to determine which are 
within a range of said simulator, as determined by said 
location data, to obtain a set of range-data elements; 

sorting said range-data elements to form a list of 
data elements in two-dimensional order; and 

spatially distributing said rage-data elements in 
accordance with spatial elements to be processed by said 
simulator, wherein said spatial elements are doppler bins. 



WO 96/16379 



PCTAJS95/14919 



34 



7. A method of using a computer to provide data 
representing weather effects to a decision-making object 
simulator, comprising the steps of: 

accessing a real-world weather database to obtain a 
multi-dimensional set of data elements, each data element 
having a location value and at least one weather effect 
value representing a traff icability condition; 

receiving location data from said simulator; 

culling said data elements to determine which are 
within a predetermined vicinity of said object, as 
determined by said location data, to obtain a set of 
neighboring data elements; and 

interpolating the weather effect values of said 
neighboring data elements to determine at least one weather 
effect value representing the net effect of said weather 
condition on said object. 
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